Computationally Efficient Target Classification in Multispectral Image Data with Deep Neural Networks

Detecting and classifying targets in video streams from surveillance cameras is a cumbersome, error-prone and expensive task. Often, the incurred costs are prohibitive for real-time monitoring. This leads to data being stored locally or transmitted to a central storage site for post-incident examination. The required communication links and archiving of the video data are still expensive and this setup excludes preemptive actions to respond to imminent threats. An effective way to overcome these limitations is to build a smart camera that transmits alerts when relevant video sequences are detected. Deep neural networks (DNNs) have come to outperform humans in visual classifications tasks. The concept of DNNs and Convolutional Networks (ConvNets) can easily be extended to make use of higher-dimensional input data such as multispectral data. We explore this opportunity in terms of achievable accuracy and required computational effort. To analyze the precision of DNNs for scene labeling in an urban surveillance scenario we have created a dataset with 8 classes obtained in a field experiment. We combine an RGB camera with a 25-channel VIS-NIR snapshot sensor to assess the potential of multispectral image data for target classification. We evaluate several new DNNs, showing that the spectral information fused together with the RGB frames can be used to improve the accuracy of the system or to achieve similar accuracy with a 3x smaller computation effort. We achieve a very high per-pixel accuracy of 99.1%. Even for scarcely occurring, but particularly interesting classes, such as cars, 75% of the pixels are labeled correctly with errors occurring only around the border of the objects. This high accuracy was obtained with a training set of only 30 labeled images, paving the way for fast adaptation to various application scenarios.Comment: Presented at SPIE Security + Defence 2016 Proc. SPIE 9997, Target and Background Signatures I

KRATOS: An Open Source Hardware-Software Platform for Rapid Research in LPWANs

Long-range (LoRa) radio technologies have recently gained momentum in the IoT landscape, allowing low-power communications over distances up to several kilometers. As a result, more and more LoRa networks are being deployed. However, commercially available LoRa devices are expensive and propriety, creating a barrier to entry and possibly slowing down developments and deployments of novel applications. Using open-source hardware and software platforms would allow more developers to test and build intelligent devices resulting in a better overall development ecosystem, lower barriers to entry, and rapid growth in the number of IoT applications. Toward this goal, this paper presents the design, implementation, and evaluation of KRATOS, a low-cost LoRa platform running ContikiOS. Both, our hardware and software designs are released as an open- source to the research community.Comment: Accepted at WiMob 201

Evaluating Spiking Neural Network On Neuromorphic Platform For Human Activity Recognition

Energy efficiency and low latency are crucial requirements for designing wearable AI-empowered human activity recognition systems, due to the hard constraints of battery operations and closed-loop feedback. While neural network models have been extensively compressed to match the stringent edge requirements, spiking neural networks and event-based sensing are recently emerging as promising solutions to further improve performance due to their inherent energy efficiency and capacity to process spatiotemporal data in very low latency. This work aims to evaluate the effectiveness of spiking neural networks on neuromorphic processors in human activity recognition for wearable applications. The case of workout recognition with wrist-worn wearable motion sensors is used as a study. A multi-threshold delta modulation approach is utilized for encoding the input sensor data into spike trains to move the pipeline into the event-based approach. The spikes trains are then fed to a spiking neural network with direct-event training, and the trained model is deployed on the research neuromorphic platform from Intel, Loihi, to evaluate energy and latency efficiency. Test results show that the spike-based workouts recognition system can achieve a comparable accuracy (87.5\%) comparable to the popular milliwatt RISC-V bases multi-core processor GAP8 with a traditional neural network ( 88.1\%) while achieving two times better energy-delay product (0.66 \si{\micro\joule\second} vs. 1.32 \si{\micro\joule\second})

Poster Abstract: MagoNode++ - A Wake-Up-Radio-Enabled Wireless Sensor Mote for Energy-Neutral Applications

The combination of low-power design, energy harvesting and ultra-low-power wake-up radios is paving the way for perpetual operation of Wireless Sensor Networks (WSNs). In this work we present the MagoNode++, a novel WSN platform supporting energy harvesting and radio-triggered wake ups for energy- neutral applications. The MagoNode++ features an energy- harvesting subsystem composed by a light or thermoelectric harvester, a battery manager and a power manager module. It further integrates a state-of-the-art RF Wake-Up Receiver (WUR) that enables low-latency asynchronous communication, virtually eliminating idle listening at the main transceiver. Experimental results show that the MagoNode++ consumes only 2.8uA with the WUR in idle listening and the rest of the platform in sleep state, making it suitable for energy-constrained WSN scenarios and for energy-neutral applications

An Accurate EEGNet-based Motor-Imagery Brain-Computer Interface for Low-Power Edge Computing

This paper presents an accurate and robust embedded motor-imagery brain-computer interface (MI-BCI). The proposed novel model, based on EEGNet, matches the requirements of memory footprint and computational resources of low-power microcontroller units (MCUs), such as the ARM Cortex-M family. Furthermore, the paper presents a set of methods, including temporal downsampling, channel selection, and narrowing of the classification window, to further scale down the model to relax memory requirements with negligible accuracy degradation. Experimental results on the Physionet EEG Motor Movement/Imagery Dataset show that standard EEGNet achieves 82.43%, 75.07%, and 65.07% classification accuracy on 2-, 3-, and 4-class MI tasks in global validation, outperforming the state-of-the-art (SoA) convolutional neural network (CNN) by 2.05%, 5.25%, and 5.48%. Our novel method further scales down the standard EEGNet at a negligible accuracy loss of 0.31% with 7.6x memory footprint reduction and a small accuracy loss of 2.51% with 15x reduction. The scaled models are deployed on a commercial Cortex-M4F MCU taking 101ms and consuming 4.28mJ per inference for operating the smallest model, and on a Cortex-M7 with 44ms and 18.1mJ per inference for the medium-sized model, enabling a fully autonomous, wearable, and accurate low-power BCI

Design and Implementation of an RSSI-Based Bluetooth Low Energy Indoor Localization System

Indoor Positioning System (IPS) is a crucial technology that enables medical staff and hospital managements to accurately locate and track persons or assets inside the medical buildings. Among other technologies, Bluetooth Low Energy (BLE) can be exploited for achieving an energy-efficient and low-cost solution. This work presents the design and implementation of an received signal strength indicator (RSSI)-based indoor localization system. The paper shows the implementation of a low complex weighted k-Nearest Neighbors algorithm that processes raw RSSI data from connection-less iBeacon's. The designed hardware and firmware are implemented around the low-power and low-cost nRF52832 from Nordic Semiconductor. Experimental evaluation with the real-time data processing has been evaluated and presented in a 7.2 m by 7.2 m room with furniture and 5 beacon nodes. The experimental results show an average error of only 0.72 m in realistic conditions. Finally, the overall power consumption of the fixed beacon with a periodic advertisement of 100 ms is only 50 uA at 3 V, which leads to a long-lasting solution of over one year with a 500 mAh coin battery.Comment: This article has been accepted for publication in the proceedings of the 2021 IEEE International Conference on Wireless and Mobile Computing, Networking And Communications (WiMob). DOI: 10.1109/WiMob52687.2021.960627

Fully Automatic Gym Exercises Recording: An IoT Solution

In recent years, working out in the gym has gotten increasingly more data-focused and many gym enthusiasts are recording their exercises to have a better overview of their historical gym activities and to make a better exercise plan for the future. As a side effect, this recording process has led to a lot of time spent painstakingly operating these apps by plugging in used types of equipment and repetitions. This project aims to automate this process using an Internet of Things (IoT) approach. Specifically, beacons with embedded ultra-low-power inertial measurement units (IMUs) are attached to the types of equipment to recognize the usage and transmit the information to gym-goers and managers. We have created a small ecosystem composed of beacons, a gateway, smartwatches, android/iPhone applications, a firebase cloud server, and a dashboard, all communicating over a mixture of Bluetooth and Wifi to distribute collected data from machines to users and gym managers in a compact and meaningful way. The system we have implemented is a working prototype of a bigger end goal and is supposed to initialize progress toward a smarter, more efficient, and still privacy-respect gym environment in the future. A small-scale real-life test shows 94.6\% accuracy in user gym session recording, which can reach up to 100\% easily with a more suitable assembling of the beacons. This promising result shows the potential of a fully automatic exercise recording system, which enables comprehensive monitoring and analysis of the exercise sessions and frees the user from manual recording. The estimated battery life of the beacon is 400 days with a 210 mAh coin battery. We also discussed the shortcoming of the current demonstration system and the future work for a reliable and ready-to-deploy automatic gym workout recording system

Self-sustaining Ultra-wideband Positioning System for Event-driven Indoor Localization

Smart and unobtrusive mobile sensor nodes that accurately track their own position have the potential to augment data collection with location-based functions. To attain this vision of unobtrusiveness, the sensor nodes must have a compact form factor and operate over long periods without battery recharging or replacement. This paper presents a self-sustaining and accurate ultra-wideband-based indoor location system with conservative infrastructure overhead. An event-driven sensing approach allows for balancing the limited energy harvested in indoor conditions with the power consumption of ultra-wideband transceivers. The presented tag-centralized concept, which combines heterogeneous system design with embedded processing, minimizes idle consumption without sacrificing functionality. Despite modest infrastructure requirements, high localization accuracy is achieved with error-correcting double-sided two-way ranging and embedded optimal multilateration. Experimental results demonstrate the benefits of the proposed system: the node achieves a quiescent current of $47~nA$ and operates at $1.2~\mu A$ while performing energy harvesting and motion detection. The energy consumption for position updates, with an accuracy of $40~cm$ (2D) in realistic non-line-of-sight conditions, is $10.84~mJ$. In an asset tracking case study within a $200~m^2$ multi-room office space, the achieved accuracy level allows for identifying 36 different desk and storage locations with an accuracy of over $95~{\%}$. The system`s long-time self-sustainability has been analyzed over $700~days$ in multiple indoor lighting situations

A Fast and Accurate Optical Flow Camera for Resource-Constrained Edge Applications

Optical Flow (OF) is the movement pattern of pixels or edges that is caused in a visual scene by the relative motion between an agent and a scene. OF is used in a wide range of computer vision algorithms and robotics applications. While the calculation of OF is a resource-demanding task in terms of computational load and memory footprint, it needs to be executed at low latency, especially in robotics applications. Therefore, OF estimation is today performed on powerful CPUs or GPUs to satisfy the stringent requirements in terms of execution speed for control and actuation. On-sensor hardware acceleration is a promising approach to enable low latency OF calculations and fast execution even on resource-constrained devices such as nano drones and AR/VR glasses and headsets. This paper analyzes the achievable accuracy, frame rate, and power consumption when using a novel optical flow sensor consisting of a global shutter camera with an Application Specific Integrated Circuit (ASIC) for optical flow computation. The paper characterizes the optical flow sensor in high frame-rate, low-latency settings, with a frame rate of up to 88 fps at the full resolution of 1124 by 1364 pixels and up to 240 fps at a reduced camera resolution of 280 by 336, for both classical camera images and optical flow data.Comment: Accepted by IWASI 202

A Survey of Multi-Source Energy Harvesting Systems

Energy harvesting allows low-power embedded devices to be powered from naturally-ocurring or unwanted environmental energy (e.g. light, vibration, or temperature difference). While a number of systems incorporating energy harvesters are now available commercially, they are specific to certain types of energy source. Energy availability can be a temporal as well as spatial effect. To address this issue, ‘hybrid’ energy harvesting systems combine multiple harvesters on the same platform, but the design of these systems is not straightforward. This paper surveys their design, including trade-offs affecting their efficiency, applicability, and ease of deployment. This survey, and the taxonomy of multi-source energy harvesting systems that it presents, will be of benefit to designers of future systems. Furthermore, we identify and comment upon the current and future research directions in this field